ECIR KEYNOTE REPORT IR Research: Systems, Interaction, Evaluation and Theories1

The practical goal of information retrieval (IR) research is to create ways to support humans to better access information in order to better carry out their tasks. Because of this, IR research has a primarily technological interest in knowledge creation – how to interact with information (better)? IR research therefore has a constructive aspect (to create novel systems) and an evaluative aspect (are they any good?). Evaluation of IR effectiveness is guided by a theory on factors that affect effectiveness. True science is about theory development, i.e., understanding and explaining, making hypotheses and testing them. Theories express structured explanatory relationships between variables such as “type of document indexing” and “quality of ranking measured by MAP”. Theories are the better, the wider range of phenomena they are able cover accurately. The paper argues that most existing theories of IR are focused on a narrow scope, theories of ranking. To fulfill its task of supporting human information access, theories that go beyond the evaluation of ranking are highly desired but face many challenges. We discuss three additional types of IR theories: theories of searching, theories of information access, and theories of information interaction.